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SUMMARY  PAGE 


THE  PROBLEM 

Psychiatric  care  is  not  available  onboard  submarines.  In  the  stressful  environment  of  a 
submarine  on  patrol,  it  is  important  that  the  mental  health  of  the  crew  be  of  the  highest  level. 
Psychiatric  screening  for  submarine  service  is,  therefore,  required. 

THE  FINDINGS 

The  Subscreen  test  for  psychiatric  screening  of  enlisted  prospective  submariners  successfully 
identifies  approximately  15%  of  psychiatric  drops  from  Basic  Enlisted  Submarine  School 
(BESS).  Further  research  is  needed  to  refine  the  test  and  to  test  its  effectiveness  for  prediction 
beyond  BESS. 

THE  APPLICATION 

Improved  psychiatric  screening  for  the  submarine  service. 


ADMINISTRATIVE  INFORMATION 

This  investigation  was  conducted  under  Naval  Medical  Research  and  Development  Command 
Research  Work  Unit  5306,  NAVSEA  Work  Request  N0002493WR01 142,  "Subscreen."  The 
views  expressed  in  this  report  are  those  of  the  authors  and  do  not  reflect  the  official  policy  or 
position  of  the  Department  of  the  Navy,  Department  of  Defense,  or  the  U.S.  government.  This 
report  was  approved  for  publication  on  16  Oct  1993  and  designated  Naval  Submarine  Medical 
Research  Report  1193. 
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ABSTRACT 


A  brief  overview  of  25  years  of  psychiatric  screening  of  enlisted  service  members  for  subma¬ 
rine  service  in  the  United  States  Navy  is  presented.  Current  screening  consists  of  administration 
of  a  submarine  specific  test,  Subscreen,  to  all  prospective  Basic  Enlisted  Submarine  School 
(BESS)  candidates.  Figures  from  1991  and  1992  show  that  approximately  9.7%  of  these  candi¬ 
dates  are  referred  for  more  extensive  testing  at  the  Psychiatry  Department  of  the  Naval  Hospital 
Groton.  Less  that  two  percent  of  all  candidates  are  subsequently  dropped  from  Bess.  Two  meas¬ 
ures  of  test  performance,  sensitivity  (the  ability  of  a  test  to  yield  a  positive  finding  when  the  indi¬ 
vidual  tested  actually  has  the  condition  being  tested  for)  and  specificity  (the  ability  of  a  test  to 
yield  a  negative  finding  when  the  individual  tested  does  not  have  the  condition)  were  studied. 
Both  sensitivity  (75%)  and  specificity  (92%)  were  very  good  for  Subscreen,  especially  compared 
to  a  test  the  USAF  administers  for  similar  purposes.  Although  some  small  changes  will  improve 
test  performance,  in  general.  Subscreen  remains  an  effective  tool  for  screening  for  Submarine 
Service. 
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PSYCHIATRIC  SCREENING  FOR  THE  SUBMARINE  SERVICE: 
ENLISTED  PERSONNEL 


History  of  subscreen 

A  submarine  on  patrol  is  a  closed  environ¬ 
ment  with  limited  access  to  the  outside  world 
for  prolonged  periods.  Onboard  medical  care 
is  available  to  treat  most  mild  to  moderate 
medical  conditions.  Psychiatric  care  is,  how¬ 
ever,  not  available.  In  the  sometimes  stressful 
environment  of  an  operating  submarine,  it  is 
important  that  the  mental  health  of  both  pro¬ 
spective  and  actual  submariners  be  of  the  high¬ 
est  levels  (Weybrew  and  Noddin,  1979). 
Psychiatric  screening  for  submarine  service 
was,  therefore,  instituted  early  in  the  develop¬ 
ment  of  the  submarine  service. 

The  early  history  of  screening  for  submarine 
duty  has  been  described  by  Weybrew  and 
Youniss  (1959).  In  the  1940s  testing  con¬ 
sisted  of  the  Shock  Fusion  Time  Test,  New 
London  NDRC  Confidential  Questionnaire, 
the  NRC  Neurotic  Inventory  and  the  Personal 
Inventory  (NDRC  Project  44,  Div.  7, 1943). 
The  first  three  of  these  were  eliminated  due  to 
length  of  presentation,  lack  of  specificity  or 
low  reliability.  The  Personal  Inventory 
showed  low  discriminability  between  normal 
and  "unsuitable"  individuals  (Weybrew  and 
Youniss,  1959;  Weybrew,  1959).  Weybrew 
and  Youniss,  therefore,  developed  the  Per¬ 
sonal  Inventory  Barometer  (PIB)  as  a  psychi¬ 
atric  screening  tool  (Weybrew  and  Youniss, 
1959;  Weybrew,  1959).  This  test  used  all  of 
the  questions  from  the  Taylor  Manifest  Anxi¬ 
ety  Scale  (Taylor,  1953),  but  changed  the  re¬ 
sponse  format  to  multiple  choice  rather  than 
True-False  responding.  Additional  questions 
were  added  to  assess  internal  validity,  atti¬ 
tudes  toward  submarine  life  and  duty,  motiva¬ 
tion,  suicide  ideation,  and  depression. 

The  Personal  Inventory  Barometer  became 
the  single  psychiatric  screening  tool  used  at 


the  U.S.  Naval  Submarine  School  for  subma¬ 
rine  candidates  (Weybrew  and  Noddin,  1969). 
The  PIB,  virtually  unchanged,  was  given  to 
all  Basic  Enlisted  Submarine  School  (BESS) 
students  for  approximately  25  years.  Demo¬ 
graphics,  ethical,  and  social  characteristics  of 
society  are  always  in  a  process  of  change. 
Definitions  of  psychological  health  also  have 
changed  over  the  last  quarter  century.  In  the 
early  1980’s  it  was  recognized  that,  due  to  its 
age,  the  PIB  might  no  longer  be  a  valid  meas¬ 
ure  of  psychological  adjustment  in  incoming 
students.  Work  began  to  develop  a  new 
screening  test  At  the  same  time,  an  analysis 
of  causes  for  premature  loss  of  non-nuclear 
trained  individuals  from  submarine  service 
was  also  undertaken  (Bryant,  1986).  This 
study  showed  that  early  losses  were  related  to 
problems  in  BESS,  middle  losses  were  related 
to  behavioral  and  psychological  problems  dur¬ 
ing  deployment  and  later  losses  (greater  than 
3  years  service)  were  related  to  misconduct. 
Motivation  was  not  considered. 

Bryant  and  Noddin  developed  a  test  that  came 
to  be  known  as  Subscreen.  Subscreen  cur¬ 
rently  incorporates  sixty  percent  of  the  ques¬ 
tions  from  the  original  PIB  but  also  includes 
additional  questions  from  the  literature  on 
stress  and  adjustment  and  questions  devel¬ 
oped  after  consultation  with  BESS  instructors 
and  psychologists  at  the  Naval  Hospital  Psy¬ 
chiatric  Clinic.  Subscreen  and  the  PIB  were 
administered  simultaneously  for  several 
BESS  classes.  After  this  testing  showed  that 
both  tests  recommended  the  same  people  for 
further  psychiatric  evaluation  and  were  there¬ 
fore  redundant,  use  of  the  PIB  was  discontin¬ 
ued.  Test-retest  reliability  for  the  PIB  was  .80 
(Weybrew  and  Youniss,  1959);  test-retest  reli¬ 
ability  measures  for  Subscreen  are  unavail¬ 
able.  One  reason  for  reviewing  the 
development  and  testing  of  Subscreen  is  that 


most  of  this  work  has  been  undocumented  or 
unpublished.  In  the  text  that  follows,  the 
original  sources  of  information  are  included 
and  noted  as  unpublished. 

Subscreen  is  given  to  all  BESS  students  the 
first  week  they  are  at  Submarine  School.  It 
takes  approximately  one  hour  to  administer  to 
the  students.  It  is  used  to  screen  individuals 
for  psychiatric  problems  that  would  preclude 
successful  service  in  the  submarine  force.  Stu¬ 
dents  whose  scores  are  above  established  lim¬ 
its  are  referred  to  the  Psychiatry  Department 
of  the  Naval  Hospital  Groton  for  psychiatric 
evaluation.  Approximately  nine  percent  of 
the  students  are  referred.  Following  this 
evaluation  the  psychologist  gives  a  recommen¬ 
dation  to  the  Medical  Department  of  the  Sub¬ 
marine  School.  BESS  1992  summary  records 
show  that  individuals  may  be  recommended 
for  return  to  BESS  (75%),  transfer  to  surface 
fleet  (9.1%),  or  discharge  (15.9%)  from  the 
Navy. 

THE  SUBSCREEN  TEST 

Subscreen  contains  240  items  scored  on  5 
scales  comprised  of  28  independent  subscales 
and  procedural  scores.  Items  that  are  highly 
correlated  and  assess  particular  aspects  of 
functioning,  such  as  affect  or  socialization 
comprise  a  scale.  The  scales  and  subscales  are 
listed  in  Appendix  A. 

The  individual  items  are  scored  with  five  re¬ 
sponse  categories.  Four  points  are  assigned 
for  strongly  agree,  3  for  agree,  2  for  disagree 
and  1  for  strongly  disagree.  Questions  that 
are  worded  in  a  negative  manner  are  reverse 
scored.  Neutral  responses  are  evaluated  sepa¬ 
rately.  As  of  1992,  all  scores  for  an  individ¬ 
ual  are  converted  to  Z  scores  based  on 
population  means  and  standard  deviations 
from  approximately  6000  BESS  students. 

Prior  to  that  date,  Z  scores  were  based  on 
means  and  standard  deviations  from  approxi¬ 


mately  800  BESS  students  tested  in  1986.  A 
Z  score  is  a  measure  of  how  far  an  individual 
score  deviates  from  the  average  answer. 

The  scoring  also  includes  further  information 
concerning  unusual  response  profiles,  missing 
responses  and  indicates  individuals  who  re¬ 
spond  in  the  affirmative  to  questions  relating 
to  suicide  and  negative  attitudes  toward  use  of 
nuclear  weapons.  Two  other  scores  measure 
how  an  individual  describes  himself.  One, 
"Good  Impression",  contains  items  typically 
endorsed  by  someone  trying  to  impress  oth¬ 
ers.  The  other,  "Distortion",  a  lie  score,  is 
also  included  on  the  scoring  protocol.  Factor 
scores  are  derived  for  Vulnerability,  Motiva¬ 
tion,  Depression-suicide,  Efficacy  (compe¬ 
tence),  Manipulation  and  Social  Isolation. 
These  factors  are  also  listed  in  Appendix  A. 
The  "Z"  attached  to  each  subscale  score  used 
in  these  factors  indicates  the  individual’s  Z 
score  used  in  the  calculation. 

Until  early  1994,  referrals  to  the  Psychiatric 
Clinic  were  determined  by  a  single  individual 
(Mr.  Noddin)  with  considerable  experience  in 
this  area.  In  his  absence,  all  Subscreen  results 
were  sent  to  the  Psychiatric  Clinic  where  a 
clinical  psychologist  reviews  them.  Heaton 
(unpublished,  1991)  investigated  whether  a 
personal  computer  based  expert  system  could 
be  developed  to  follow  the  same  rules  that 
were  used  by  Mr.  Noddin  to  decide  which 
cases  to  refer.  She  was  able  to  develop  a  set 
of  31  rules  which  could  reproduce  referrals 
with  a  high  degree  of  both  sensitivity  (98.9%) 
and  specificity  (96.8%).  These  rules  are  ap¬ 
plied  to  the  answers  to  two  questions,  the  Z 
scores  of  15  subscales  and  the  percentage  of 
missing  and  neutral  responses.  None  of  the 
submarine  specific  motivation  subscales  were 
used  as  a  basis  for  referral  by  the  screening 
psychologist  and  none  are  included  in  the  ex¬ 
pert  system.  The  rales  developed  by  Heaton 
are  the  current  basis  for  referral  by  the 
NSMRL  tester.  A  computer  decision  algo- 
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rithm  has  not  been  implemented  in  the  scoring 
routine.  An  on-going  NSMRL  project  is  com¬ 
paring  an  expert  system  to  a  neural  nets  ap¬ 
proach  for  determining  referrals. 

The  Psychiatry  Department  at  the  Naval  Hos¬ 
pital  Groton  considered  using  the  Minnesota 
Multiphasic  Personality  Inventory  (MMPI)  or 
another  commercial  test  (J.  Wallace,  person¬ 
nel  communication,  1992)  and  is  satisfied 
with  Subscreen  as  a  screening  test.  Interview 
of  the  clinical  psychologist  at  the  Psychiatry 
Department  revealed  that  using  standard  scor¬ 
ing  protocols  the  MMPI  yielded  "too  high"  a 
rate  of  referrals  to  the  department  and  also 
took  much  longer  to  administer  (2-3  hours) 
than  Subscreen.  A  revision  of  the  original 
MMPI,  the  MMPI-2,  is  currently  used  at  the 
Psychiatry  Department  as  a  clinical  instru¬ 
ment  to  provide  further  information  to  the  cli¬ 
nician  on  referred  individuals.  The  MMPI 
has  also  been  found  by  the  Canadian  Forces 
to  be  inappropriate  as  an  assessment  tool  for 
psychological  fitness  of  applicants  for  Cana¬ 
dian  Submarine  service  (Okros,  1989)  for  the 
same  reasons.  The  same  conclusion  was 
reached  by  the  US  Air  Force  (Bloom,  1983). 
While  use  of  revised  norms  would  reduce  the 
number  referred,  it  would  still  take  longer  to 
administer  and  score  the  tests.  Other  estab¬ 
lished  tests  such  as  the  California  Psychologi¬ 
cal  Inventory,  Taylor  Manifest  Anxiety  Test 
and  the  Gordon  Personal  Profile  have  not 
been  tested  for  this  specific  use.  The  Air 
Force  and  the  Navy  are  currently  using  the 
NEO-Personality  Test  Five  Factor  Inventory 
(Crawford  and  Fiedler,  1991)  as  the  second 
step  of  basic  recruit  screening.  Another  impor¬ 
tant  consideration  is  that  the  MMPI-2  and 
other  commercial  tests  do  not  address  either 
Navy  or  submarine  specific  motivation  issues. 

Reliability  of  subscreen  subscales 

Katz  and  Rexer  (unpublished,  1990)  assem¬ 
bled  a  data  set  of  approximately  6000  men 


who  took  Subscreen  at  BESS  between  June 
1986  and  February  1990.  The  internal  consis¬ 
tency  of  the  subscales  was  assessed.  New 
means  and  standard  deviations  for  the 
subscales  were  also  calculated.  The  reliability 
of  subscales  is  measured  by  performing  an 
item  analysis  on  the  components  of  the 
subscale.  This  analysis  determines  to  what  de¬ 
gree  the  items  that  make  up  a  subscale  are  re¬ 
lated  (correlated).  Cronbach’s  alpha, 
equivalent  to  the  average  split-half  correla¬ 
tion,  was  used  as  the  measure  of  reliability  of 
the  subscales  (Cronbach,  1951).  Alpha  scores 
can  range  from  0  to  1.0.  The  higher  the  num¬ 
ber  the  greater  the  reliability.  Table  1  shows 
the  mean,  standard  deviations,  number  of 
questions  in  the  subscale  and  the  alpha  statis¬ 
tics  for  each  subscale. 

These  data  show  that  most  of  the  subscales 
have  reasonably  high  reliability  (over  .70). 
Only  three  subscales.  Uncertain  about  Subma¬ 
rines  (UNS),  Problems  Submarine  School 
(PSS),  and  Self-Criticism  (SCR),  showed  low 
reliability.  One  subscale,  Coercive  Attitudes 
(COE),  showed  a  large  number  of  failures  to 
respond.  Five  other  subscales  showed  prob¬ 
lems  with  one  or  two  items.  These  were  Un¬ 
conditional  Acceptance  of  Submarines  (UCS), 
Social  Isolate  (SOI),  Impulsive  (IMP),  De¬ 
pendency  (DEPND),  and  Competency 
(COM).  Katz  and  Rexer  conclude  these  could 
all  be  improved  by  deletion  or  modification  of 
these  problem  items.  This  process  would  in¬ 
crease  the  reliability  of  the  associated 
subscales  and  not  affect  the  overall  utility  of 
Subscreen. 

Katz  and  Rexer  (unpublished,  1990)  also 
found  differences  between  the  means  and 
standard  deviations  for  subscales  obtained  for 
the  original  1986  sample  and  for  the  larger 
sample  they  used.  Ten  of  the  subscales 
showed  response  patterns  that  suggested  that 
the  original  scoring  program  should  be  re¬ 
vised.  The  Z  scores  for  each  subscale  were 
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calculated  using  the  original  means  and  stand¬ 
ard  deviations.  If  the  original  values  were 
still  accurate  the  mean  Z  scores  for  each 
subscale  for  the  new  sample  should  be  zero. 
Rather  small  departures  were  found  for  some 
subscales  while  considerably  larger  changes 


were  found  for  others.  Departures  from  a  Z 
equal  to  zero  were  found  for  Impulsive  (IMP) 
(.144),  Social  Isolate  (SOI)  (.056),  Nervous  or 
Worrying  (ANX)  (.054),  Unusual  Thoughts 
(BTH)  (-.062),  Claustrophobic  Feelings 
(CLA)  (-.063),  Uncertainty  About  Subs 


Table  1 


Reliability  for  Subscreen  Subscales 


Subscale 

Mean  (M) 

SD 

No.  of 
Questions 

Alpha 

Reliability 

Procedural 

Distortion 

23.34 

3.0 

8 

.71 

Good  Impression 

38.8 

3.7 

12 

.79 

Submarine  Specific  Motivation 

Mistake  Joining  Subs 

13.9 

3.5 

8 

.80 

Uncertain  About  Subs* 

14.2 

2.4 

6 

.46 

Conditional  Acceptance  Subs 

9.6 

2.4 

5 

.72 

Unconditional  Acceptance  Subs*** 

24.7 

3.5 

8 

.76 

Problems  Submerging 

7.9 

1.8 

4 

.67 

Problems  Submarine  School* 

14.4 

1.6 

o 

-.01 

Affective 

Physical  Well  Being 

15.0 

9.2 

9 

.73 

Low  Situational  Control 

21.5 

3.8 

10 

.76 

Nervous  or  Worrying 

24.6 

4.0 

11 

.77 

Depressed  Mood 

16.1 

3.9 

9 

.86 

Socialization 

Coercive  Attitudes** 

28.6 

4.1 

14 

.72 

Aggressive/Destructive 

17.11 

3.2 

9 

.73 

Problems  Home/School 

20.8 

4.2 

11 

.76 

Social  Isolate*** 

25.3 

3.7 

12 

.63 

Impulsive*** 

27.7 

3.8 

12 

9 

.65 

Social  Support 

28.1 

3.3 

.68 

Additional 

Unusual  Thoughts 

15.5 

3.0 

8 

.73 

Unusual  Physical  Complaints 

9.6 

2.2 

6 

.77 

Suicidal  Thoughts 

9.1 

2.4 

6 

.78 

Claustrophobic  Feelings 

11.5 

2.3 

6 

.74 

Problems  Nuclear 

10.1 

2.5 

6 

.63 

Dependency*** 

25.7 

3.6 

11 

Q 

.67 

Self-Criticism* 

21.6 

2.3 

y 

n 

.14 

Competency*** 

21.6 

2.4 

/ 

.60 

*  Subscales  with  low  reliability 
**  Subscale  with  failures  to  respond 

***  Problems  with  one  or  two  specific  questions  used  in  subscale 
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(UNS)  (.398),  Problems  Nuclear  (NUC) 

(.071),  Problems  Submarine  School  (PSS) 
(.384),  Dependency  (DEPND)  (-.063),  and 
Self-Criticism  (SCR)  (-.134).  The  new  means 
and  standard  deviations  were  incorporated 
into  the  scoring  program  currently  in  use  in 
1992. 

Subscreen  and  Psych  Drops 
Overall  Performance 

The  majority  of  submarine  candidates  volun¬ 
teer  for  submarine  service  either  at  the  time 
they  are  recruited  or  following  completion  of 
Basic  Recruit  Training.  Volunteers  are  also 
accepted  from  other  segments  of  the  Navy. 
Most  BESS  students,  therefore,  are  younger 
than  20  years  of  age  and  in  the  lowest  three 
military  paygrades.  Losses  from  BESS  occur 
due  to  psychiatric  disqualification  and  for 
medical,  academic  and  motivational  reasons. 

Subscreen  was  developed  to  screen  BESS  stu¬ 
dents  for  motivational,  affective  and  social 
problems  that  would  preclude  successful  serv¬ 
ice  in  the  submarine  force.  Although  the  test 
was  developed  with  numerous  scales,  some  of 


these  scales  (motivation)  are  not  used  in  the 
decision  process.  Only  three  scales  and  a  few 
additional  items  on  Subscreen  are  currently 
used  by  NSMRL  to  refer  students  to  the  Psy¬ 
chiatric  Department.  These  scales  are  com¬ 
posed  of  questions  that  measure  affect  and 
socialization.  Subscales  that  measure  unusual 
thoughts  (bizarre  cognition),  unusual  physical 
complaints,  and  claustrophobia  are  also  used. 
These  are  listed  under  additional  subscales  in 
Appendix  A.  Positive  answers  to  questions 
concerning  suicide  and  negative  feelings 
about  serving  in  a  force  with  nuclear  weapons 
are  also  immediate  grounds  for  referral.  Al¬ 
though  part  of  the  scoring  protocol  includes 
factor  scores  with  the  weights  as  determined 
by  multiple  regression  analysis,  these  are  not 
currently  used  in  the  decision  process  for  re¬ 
ferral. 

Figure  1  shows  the  results  of  the  multiple  re¬ 
gression  analysis  performed  by  Katz  and 
Rexer  (1990)  to  predict  psychiatric  drop 
status  from  BESS  in  a  cohort  of  approxi¬ 
mately  650  of  the  more  recent  cases  from  the 
same  file  used  to  analyze  the  reliability  of  the 
subscales.  The  coefficient  of  determination, 


BESS  PSYCH  DROP  =  0  -  NORMAL 

1  -  PSYCH  DROP 

AFFECTIVE  VARIABLE  SCORE  r2  =  .  128 

df= 

484 

PSYCH  DROP  =  .029  +  .031  (PWL)  +  .028  (SUI)  + 

.014  (LOC)  -  .015  (BTH) 

MOTIVATION  VARIABLE  SCORE  r2  =  .146 

rf/ =  486 

PSYCH  DROP  =  .033  -  .031  (UCS)  +  .022  (NUC)  + 

.025  (MJS) 

SOCIALIZATION  SCORE  r2  =  .074 

df= 

=  501 

PSYCH  DROP  =  .029  +  .027  (PHS)  +  .024  (SOI) 

Figure  1.  Multiple  regression  to  predict  BESS  psych  drops  (Katz  and  Rexer,  1990). 


5 


r2,  ranges  from  0  to  1.0.  It  is  close  to  1.0  if 
the  variables  in  the  variable  set  are  good  pre¬ 
dictors  of  psychiatric  drop  status.  The  Affec¬ 
tive,  Motivation  and  Socialization  results  for 
Subscreen  are  shown.  These  r2 s  reflect  the 
criteria  used  to  refer  individuals  for  evalu¬ 
ation.  Because  the  affective  and  socialization 
scores  are  primarily  used  for  these  referrals 
they  show  the  highest  predictive  values;  be¬ 
cause  motivational  scores  are  not  used  as  a  ba¬ 
sis  for  referral,  they  show  the  lowest.  These 
results  should  be  looked  at  somewhat  criti¬ 
cally  as  only  13  psychiatric  drops  were  in¬ 
cluded  in  the  analysis.  As  would  be  expected 
the  r2  are  low  under  these  circumstances. 

The  Naval  Submarine  Medical  Research  Labo¬ 
ratory  (Schlichting  and  Noddin,  unpublished, 
1992)  recently  evaluated  all  psychiatric  drops 
from  BESS  for  FY91  and  FY92.  A  total  of 
3360  students  were  screened  using  Subscreen 
in  FY91;  2166  in  FY92.  Slightly  less  than 
10%  were  referred  to  the  Psychiatry  Depart¬ 
ment  for  evaluation;  1.7%  of  the  total  5526 
were  dropped  through  the  use  of  Subscreen. 

An  additional  .6%  were  dropped  through 
other  referrals.  Subscreen  referrals  constitute 
75%  of  the  psychiatric  drops  from  BESS  for 
the  two  years.  Comparable  figures  for  AF- 
MET,  the  Air  Force  Screening  program,  show 
that  only  22%  of  the  Psychiatric  drops  during 
the  first  six  weeks  of  USAF  training  were 
identified  by  their  testing  protocol  (Crawford 
and  Fiedler,  1991;  E.  Fiedler  personal  commu¬ 
nication,  1992).  The  final  percentage  of  re¬ 
cruits  dropped  for  psychiatric  reasons  is  much 
lower,  .4%,  for  Air  Force  basic  training  ver¬ 
sus  2.3%  for  Submarine  School  students. 

To  evaluate  screening  tests  one  can  compute 
several  measures  of  test  performance  (Wool- 
son,  1987).  Sensitivity  is  the  ability  of  a  test 
to  yield  a  positive  finding  when  the  individual 
tested  actually  has  the  condition  for  which  he 
or  she  is  being  tested.  Specificity  is  the  abil¬ 
ity  of  the  test  to  yield  a  negative  finding  when 


the  individual  does  not  have  the  condition. 
Different  screening  levels  or  cutoff  scores  for 
what  is  considered  a  positive  finding  can  be 
developed  depending  on  the  relative  impor¬ 
tance  of  detecting  the  condition  and  falsely  de¬ 
claring  the  condition.  Positive  predictive 
value  of  a  test  is  the  probability  that  a  person 
who  has  a  positive  test  has  the  condition. 
Prevalence  of  the  condition  in  the  population 
affects  positive  predictive  value.  Negative 
predictive  value  is  the  probability  that  a  per¬ 
son  who  has  a  negative  test  does  not  have  the 
condition.  Appendix  B  contains  a  more  com¬ 
plete  description  of  these  measures  and  their 
calculation. 

The  combined  Subscreen  data  for  FY91  and 
FY92  described  above  showed  a  sensitivity  of 
75%,  a  specificity  of  92%,  a  positive  predic¬ 
tive  value  of  17.4%,  and  a  negative  predictive 
value  of  99.4%.  Comparable  AFMET  data 
for  the  period  1975  to  1982  for  Air  Force  ba¬ 
sic  recruits  shows  a  sensitivity  of  22%,  speci¬ 
ficity  of  98.9%,  a  positive  predictive  value  of 
26.9%  and  a  negative  predictive  value  of 
98.9%.  While  Subscreen  is  better  at  identify¬ 
ing  people  with  psychiatric  problems,  AF¬ 
MET  is  better  at  identifying  people  who  do 
not  have  a  psychiatric  problem. 

Internal  Performance 

Analysis  of  variance  was  performed  on  each 
Subscreen  subscale  and  factor  for  the  FY91  re¬ 
sults  to  determine  whether  there  were  signifi¬ 
cant  differences  among  all  the  actual 
Psychiatric  drops  identified  by  Subscreen 
(n=55),  a  random  sample  of  the  individuals  re¬ 
ferred  by  Subscreen  who  were  not  dropped 
for  psychiatric  reasons  (n=101),  all  those 
dropped  for  psychiatric  reasons  who  were  not 
referred  by  Subscreen  (n=21),  and  a  random 
sample  of  those  who  were  not  referred  by  Sub¬ 
screen  and  not  dropped  for  psychiatric  rea¬ 
sons  (n=142). 
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Table  2. 

Univariate  E-Tests  with  (3  £15)  Degrees  of  Freedom 


VARIABLE 

£ 

e _ 

AGE 

.41 

.74 

EDUCATION 

2.63 

.05 

PROFILE 

20.03 

.001 

NEUTRAL% 

5.90 

.001 

MISSING% 

.34 

.79 

PROB 

49.70 

.001 

Procedural 

Distortion 

19.35 

.001 

Good  Impression 

16.53 

.001 

Classification 

46.24 

.001 

Extreme  Scores 

.22 

.88 

Submarine  Specific  Motivation 

Mistake  Joining  Subs 

62.44 

.001 

Uncertain  About  Subs 

67.42 

.001 

Conditional  Acceptance  Subs 

7.6 

.001 

Unconditional  Acceptance  Subs 

64.89 

.001 

Problems  Submerging 

54.42 

.001 

Problems  Submarine  School 

21.42 

.001 

Affective 

Physical  Well  Being 

43.39 

.001 

Low  Situational  Control 

72.36 

.001 

Nervous  or  Worrying 

53.08 

.001 

Depressed  Mood 

68.18 

.001 

Socialization 

Coercive  Attitudes 

29.27 

.001 

Aggressive/Destructive 

31.97 

.001 

Problems  Home/School 

57.8 

.001 

Social  Isolate 

29.65 

.001 

Impulsive 

32.29 

.001 

Social  Support 

24.27 

.001 

Additional 

Unusual  Thoughts 

58.46 

.001 

Unusual  Physical  Complaints 

31.31 

.001 

Suicidal  Thoughts 

65.74 

.001 

Claustrophobic  Feelings 

45.78 

.001 

Problems  Nuclear 

26.88 

.007 

Dependency 

19.13 

.001 

Self-Criticism 

24.13 

.001 

FACTORS 

Vulnerability 

77.53 

.001 

Negative  Motivation 

92.86 

.001 

Depression/S  uicide 

59.24 

.001 

Competence 

13.35 

.001 

Manipulation 

32.61 

.001 

Isolation/Social  Support 

40.51 

.001 
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Table  3. 

Subscreen  Z  Scores. 


Subscreen 

Yes 

No 

Yes 

No 

Psych  Drop 

Yes 

Yes 

No 

No 

Scale 


Procedural 


Classification 

3.13* 

.36 

.73** 

.00 

Extreme  Scores 

.12 

-.03 

.06 

.01 

Distortion  (Lie) 

-.94* 

.37 

-.50* 

.16 

Good  Impression 

-1.18* 

.09 

-.46** 

.02 

Submarine  Specific  Motivation 

Mistake  Joining  Subs 

2.68* 

.41 

.67** 

-.04 

Uncertain  About  Subs 

2.02* 

.47 

.97* 

.15 

Conditional  Submarine 

2.28* 

.79 

.58 

.30 

Uncond.  Submarine/Pro  SubSchool 

-2.21* 

.42 

-.45** 

.19 

Problems  Submerging 

1.18* 

.62 

.54 

.17 

Affective 

Physical  Well-being 

1.67* 

-.28 

.39 

-.17 

Low  Situational  Contra 

1.79* 

.06* 

1.02* 

-.21 

Nervous  or  Anxious 

1.43* 

.04 

.93* 

-.21 

Depressed  Mood 

1.91* 

.14 

.97* 

-.24 

Socialization 

Coercive  Attitudes 

1.30** 

-.03 

94** 

-.1 

Aggressive/Destructive 

1.13** 

-.01 

;g3** 

.27 

Problem  Home  or  School 

1.50* 

.14 

1.00* 

-.15 

Social  Isolate 

1.07** 

0 

.85** 

-.20 

Impulsive 

1.03** 

.10 

.96** 

-.10 

Social  Support 

-1.15** 

.11 

-.83** 

.12 

Additional 


Unusual  Thoughts 

Unusual  Physical  Complaints 

Suicide 

Claustrophobia 
Problems  with  Nuclear 
Dependency 
Self-criticism 


1.46* 

-.11 

.84* 

-.30 

1.56* 

.43 

.52 

-.05 

2.13* 

.12 

1.01* 

-.25 

1.78* 

.20 

.58 

-.23 

1.80* 

.32 

.58 

.08 

.84** 

.00 

.61** 

-.28 

.01 

-.17 

.65* 

-.35 

*  Group  differs  from  all  others  at  p  <  .05 

**  Group  differs  from  No/Y es  and  No/No  at  p  <  .05 


The  groups  did  not  differ  in  age,  education, 
number  of  neutral  responses,  number  of  ex¬ 
treme  scores  or  number  of  missing  responses. 
For  all  other  subscales  and  factors  there  were 
statistically  significant  differences  across  the 


groups  at  the  p  <  .007  or  better  level.  These 
results  are  shown  in  Table  2. 

Post  hoc  Newman  Keuls  tests  showed  that  for 
most  subscales  the  group  that  was  referred 
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and  dropped  differed  from  both  other  groups 
that  were  not  referred.  The  single  exception 
was  the  self  criticism  (SCR)  subscale.  The 
group  that  was  referred  and  dropped  did  not 
differ  from  the  group  that  was  referred  but  not 
dropped,  on  most  of  the  Socialization 
subscales  and  on  the  dependency  subscale  of 
the  Additional  Scale.  This  is  not  surprising 
because  these  subscales  are  used  as  the  basis 
of  the  referral  decision.  The  two  referred 
groups  did  differ  on  the  all  other  subscales. 
Table  3  shows  the  results  of  all  scales  and 
subscales.  Table  4  shows  the  results  for  the 
six  factors.  As  noted  above,  Appendix  A  con¬ 
tains  descriptions  of  each  of  the  subscales.  In 
each  table,  data  are  shown  for  the  group  that 
was  referred  by  Subscreen  and  dropped  for 
psychiatric  reasons  (yes/yes),  the  group  that 
was  not  referred  and  was  dropped  for  psychi¬ 
atric  reasons  (no/yes),  the  group  that  was  re¬ 
ferred  and  not  dropped  (yes/no),  and  a  group 
that  was  not  referred  and  not  dropped.  A  sin¬ 
gle  asterisk  is  used  to  denote  a  group  that  dif¬ 


fered  significantly  from  all  three  other  groups. 
A  double  asterisk  is  used  to  denote  referred 
groups  that  differed  only  from  the  two  groups 
that  were  not  referred.  For  most  subscales 
there  appears  to  a  hierarchy  of  scores.  The 
group  that  was  referred  and  dropped  shows 
the  largest  Z  scores  (that  is,  they  are  the  "most 
abnormal");  the  group  that  was  referred  but 
not  dropped  shows  intermediate  Z  scores.  The 
groups  that  were  not  referred  show  the  small¬ 
est  Z  scores  and  do  not  differ  from  each  other. 

The  results  of  the  analysis  of  variance  re¬ 
ported  above  suggest  that  a  set  of  discriminat¬ 
ing  variables  could  be  developed  that  would 
more  accurately  determine  which  individuals 
should  be  referred  to  the  Psychiatric  Clinic  for 
evaluation.  A  discriminant  analysis  (WILKS 
method)  was  performed  to  determine  if  there 
was  a  linear  combination  of  subscale,  demo¬ 
graphic  and  factor  scores  that  could  discrimi¬ 
nate  between  individuals  who  were  referred 
and  dropped  and  those  who  were  referred  and 


Table  4. 

Factor  and  other  scores  for  SUBSCREEN 


SUBSCREEN 

FACTORS 

Subscreen 

Yes 

No 

Yes 

No 

Psych  Drop 

Yes 

Yes 

No 

No 

Vulnerability 

1.54* 

-.14 

.94* 

-.37 

Motivation 

2.70* 

.26 

.72* 

-.22 

Depression/Suicide 

2.01* 

.26 

.77* 

-.11 

Competency 

-1.08** 

.02 

-.32 

.00 

Manipulative 

1.17** 

-.17 

.89** 

-.24 

Isolation  vs  Support 

1.43** 

.03 

1.14** 

-.06 

OTHER  SCORES 

Problem 

2.49* 

.26 

1.52** 

-.19 

Motivation 

3.72* 

.26 

.66** 

-.15 

Affective 

4.89* 

-.13 

1.86** 

-.24 

*  Group  differs  from  all  others  at  p  <  .05 
**  Group  differs  from  No/Y es  and  No/No  at  p  <  .05 
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not  dropped.  Table  5  shows  the  classification 
of  individuals  as  a  result  of  the  discriminant 
function  coefficients  developed  in  this  analy¬ 
sis.  This  table  shows  how  well  the  discrimi¬ 
nant  function  performs  in  discriminating 
drops  from  non-drops  in  this  sample.  Only 
students  for  whom  a  score  was  available  on 
all  subscales  are  included.  Figures  are  in¬ 
cluded  for  the  percentages  of  each  group  pre¬ 
dicted  to  fall  in  each  category  by  the  function. 

Discriminating  variables  included  Good  Im¬ 
pression,  Mistake  Joining  Subs,  Problems  in 
Subschool,  Physical  Wellbeing,  Low  Situ¬ 
ational  Control,  Nervous  (Anxiety),  Problems 
at  Home  or  School,  Suicide,  Self-criticism 
and  Vulnerability.  These  variables  showed 
excellent  discrimination  of  the  two  groups. 
Sensitivity  of  this  function  is  82.5%,  specific¬ 
ity  is  91.7%.  Positive  predictive  value  is 
70.2%  and  negative  predictive  value  is  90.6%. 
88%  of  the  total  group  were  correctly  classi¬ 
fied.  A  second  discriminant  function  analysis 
was  performed  to  see  how  well  a  function 
could  discriminate  among  those  referred  and 
dropped,  those  referred  and  not  dropped,  and 
those  not  referred  and  not  dropped.  This  func¬ 
tion  correctly  classified  83%  of  the  students. 

One  interesting  result  of  this  analysis  was  that 
the  Submarine  Specific  Motivational 


subscales  showed  better  discrimination  of 
drops  versus  non-drops  than  either  the  Affec¬ 
tive  or  Socialization  subscales  (see  Table  3). 
Students  who  were  referred  and  dropped 
show  a  mean  Z  score  of  2.0  or  greater  for  5  of 
the  6  Motivation  subscales  while  the  largest  Z 
score  for  any  of  the  Motivation  subscales  in 
the  group  that  was  referred  and  not  dropped  is 
less  than  1.0,  and  in  the  not  referred  and  not 
dropped  group,  the  largest  Z  score  is  .30. 

This  occurs  despite  the  fact  that  these 
subscales  were  not  used  as  a  basis  for  initial 
referral.  This  strongly  suggests  that  revision 
of  the  process  used  to  refer  students  to  include 
these  subscales  could  improve  the  screening 
process  by  referring  fewer  students  while 
keeping  a  similar  number  of  drops. 

One  problem  with  discriminant  function  analy¬ 
sis  is  that  it  maximizes  the  sources  of  variance 
between  groups  and  may  include  or  capitalize 
on  sources  of  variance  unique  to  the  particular 
data  set.  This  unique  variance  is  unrelated  to 
the  variables  of  interest  and  does  not  general¬ 
ize  to  a  different  data  set.  A  discriminant 
function  obtained  on  one  data  set  must  there¬ 
fore  be  tested  on  a  different  data  set  to  assure 
that  the  function  is  valid.  Another  test  proce¬ 
dure  derives  the  function  on  half  of  the  avail¬ 
able  data  and  tests  the  function  on  the  other 
half  (jack-knife  procedure).  Although  the 


iauiv  «...  ... 

Predictive  results  of  discriminant  analysis  to  discriminate  psychiatric 
drops  from  BESS  referred  by  Subscreen  with  individuals  referred  but 
not  dropped  for  psychiatric  reasons _ 

Actual  Group _ #Cases _ Predicted  Group _ 

Dropped _ Not  Dropped 


Dropped 

40 

33 

7 

(82.5%) 

(17.5%) 

Not  Dropped 

85 

8 

77 

(9.4%) 

(90.6%) 
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data  included  above  show  considerable  prom¬ 
ise  in  being  able  to  refine  Subscreen  scoring 
protocols  to  better  predict  those  individuals 
who  would  be  Psychiatric  drops,  further  re¬ 
search  with  a  larger  data  set  would  be  re¬ 
quired  to  accurately  determine  the  best  set  of 
discriminant  function  coefficients.  It  would 
also  be  desirable  to  perform  this  analysis  on  a 
group  of  submarine  school  candidates  for 
which  the  clinical  evaluations  were  made 
without  access  to  the  Subscreen  results.  These 
functions  could  be  incorporated  into  the  scor¬ 
ing  program.  New  norms  to  calculate  the  Z 
scores  also  need  to  be  computed  and  added  pe¬ 
riodically  to  the  scoring  routines  as  the  popu¬ 
lation  characteristics  change. 

Subscreen  used  to  predict 
passing  or  failing  BESS 

Although  Subscreen  was  developed  to  assess 
psychological  functioning,  the  possibility  that 
it  could  be  useful  in  predicting  attrition  from 
BESS  for  all  causes  has  also  been  studied. 
Bryant  (unpublished,  1987)  performed  a  dis¬ 
criminant  function  analysis  of  Subscreen  to 
determine  whether  it  could  be  used  to  predict 
loss  from  all  causes  from  BESS.  Bryant  ana¬ 
lyzed  Subscreen  scores  from  778  enlisted  per¬ 
sonnel  in  BESS.  Eleven  percent  (n=83)  of 
these  did  not  graduate.  This  analysis  included 
all  students  who  did  not  complete  BESS.  Psy¬ 
chological  drops  were  included  in  this  group 
but  were  not  studied  separately.  Bryant  re¬ 
ported  a  general  discriminant  function  that 
had  a  total  canonical  r2  =  .38  (p  <  .001, 
df=l  1).  This  figure  specifies  the  proportion 
of  variance  in  either  variable  which  is  linearly 
accounted  for  by  the  other.  Note  that  this  ac¬ 
counts  for  approximately  14%  of  the  variance. 
Most  of  the  variance  is,  therefore,  not  ac¬ 
counted  for  by  these  variables. 

Eleven  of  the  28  subscales  contributed  to  the 
general  prediction  equation.  Separate  func¬ 
tions  for  the  Affective,  Socialization  and  Moti¬ 


vation  scales  were  also  developed.  The  gen¬ 
eral  predictive  equation  correctly  classified 
79%  of  the  group.  As  Bryant  notes,  assigning 
all  of  the  students  to  the  pass  group  would 
have  yielded  a  correct  classification  of  89%  of 
the  group.  For  attrites,  60.2%  (n=50)  were 
correctly  identified  and  19%  (n=132)  of  the 
non-attrites  were  incorrectly  identified  as  po¬ 
tential  attrites.  These  data  show  a  sensitivity 
of  60%  and  a  selectivity  of  86%.  Positive  pre¬ 
dictive  value,  the  probability  that  someone 
with  a  loss  prediction  was  actually  lost,  is 
27%. 

Bryant  concludes  that  Subscreen  "does 
identify  individuals  who  become  attrites".  He 
also  concludes  that  to  improve  prediction 
accuracy,  further  research  is  needed  to  de¬ 
velop  the  most  appropriate  cutoff  scores  for 
the  discriminant  function.  There  are  several 
reasons  why  individuals  may  not  complete 
BESS.  These  include  not  only  psychiatric 
problems  but  also  motivational,  academic  and 
personal  reasons.  It  would  be  naive  to  expect 
that  a  single  discriminant  function  would  dis¬ 
criminate  among  all  attrites  and  non-attrites. 
Conceivably  functions  to  predict  each  type  of 
drop  from  BESS  could  be  developed,  but  the 
use  of  a  test  designed  principally  to  screen  for 
psychiatric  problems  would  seem  inappropri¬ 
ate,  and  in  this  attempt,  at  least,  was  unsuc¬ 
cessful. 

Subscreen  used  to  predict 
less  than  honorable  discharge 

In  fact,  the  possible  use  of  Subscreen  for  just 
such  a  broader  screening  has  been  studied.  In 
1991,  Katz  (unpublished)  examined  data  from 
4695  enlisted  men  who  took  the  Subscreen  in¬ 
ventory  in  1986  and  summarized  performance 
(attrition,  pay  grade)  data  obtained  from  Na¬ 
val  Personnel  Research  and  Development 
Command  (NPRDC)  and  Subscreen  results  us¬ 
ing  only  the  multiple  regression  results  deter¬ 
mined  previously  in  Katz  and  Rexer  (1990) 
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for  the  three  scales  that  best  predicted  psych 
drops.  He  did  not  perform  a  multiple  regres¬ 
sion  analysis  of  the  raw  item  scores  or  all  of 
the  scale  or  factor  scores. 

Complete  data  were  available  for  only  4675 
of  the  people.  Sixty-four  percent  were  still  on 
active  duty  in  1990.  Thirty-two  percent  failed 
to  complete  their  enlistment;  17.7%  left  with 
less  than  an  honorable  discharge. 

This  report  looked  only  at  the  following  Sub- 
screen  scores  as  predictors  of  attrition: 

1.  Affective  Factor 

2.  Socialization  Factor 

3.  Motivation  Factor 

4.  Low  Frequency  (endorses  items  rarely 
endorsed  by  the  main  population). 

The  same  regression  coefficients  determined 
in  Katz  and  Rexer  (1990)  were  used  to  evalu¬ 
ate  discharge  type.  These  were  previously 
shown  in  Figure  1.  The  Socialization  and 
Low  Frequency  scores  were  the  most  useful 
for  predicting  type  of  discharge.  For  the  So¬ 
cialization  score  the  difference  between  the 
groups  was  statistically  significant,  but 
amounted  to  only  15%  of  one  standard  devia¬ 
tion  (SD).  The  men  in  the  attrition  group  also 
endorsed  unusual  items  slightly  more  fre¬ 
quently.  This  result  is  less  predictive  than 
that  obtained  by  Bryant  for  general  Submarine 
School  attrition  or  by  Katz  and  Rexer  (1990) 
for  attrition  from  Submarine  School.  Very  lit¬ 
tle  of  the  variance  between  the  groups  is 
accounted  for  by  these  two  scores,  approxi¬ 
mately  2%.  That  means  that  98%  of  the  dif¬ 
ference  between  the  two  groups  is  not 
accounted  for  by  Subscreen  test  scores.  The 
Motivation  and  Affective  scores  did  not  distin¬ 
guish  between  the  groups  for  discharge  type. 


Katz  recommends  using  a  Socialization  Z 
score  of  1.5  or  greater  as  an  "at  risk"  criterion 
score  for  less  than  honorable  discharge.  He 
makes  this  recommendation  because  the  per¬ 
cent  of  the  individuals  obtaining  scores 
greater  than  this  is  higher  in  the  Less  than 
Honorable  discharge  groups  (mean  15.8%) 
than  in  those  still  on  active  duty  (1.9%)  and 
those  receiving  an  Honorable  discharge 
(5.7%).  The  sensitivity  (probability  of  a  Z 
score  greater  than  or  equal  to  1.5  given  that  a 
less  than  honorable  discharge  occurred)  and 
the  specificity  (the  probability  of  Z  score  less 
than  1.5  score  when  a  honorable  discharge  or 
continuance  of  active  duty  occurred)  of  this 
recommended  cutoff  were  examined.  The  sen¬ 
sitivity  is  very  low,  7.6%.  The  specificity  is 
fairly  high,  97%.  Specificity  is  high,  of 
course,  because  of  the  large  proportion  of  true 
negatives.  If  all  the  personnel  still  on  active 
duty  who  received  a  score  above  1.5  (and, 
therefore,  were  classified  in  the  above  figures 
as  false  positive)  did,  in  fact,  later  receive  less 
than  honorable  discharges)  the  sensitivity 
only  goes  up  to  23%.  In  actual  figures  using 
this  criteria  would  have  correctly  recom¬ 
mended  that  47  men  were  at  risk  of  less  than 
honorable  discharge;  94  men  would  have 
been  recommended  who  did  not  actually  re¬ 
ceive  a  less  than  honorable  discharge,  and  564 
men  who  would  not  have  been  recommended 
did,  in  fact,  receive  a  less  than  honorable  dis¬ 
charge.  The  remaining  3431  men  would  not 
have  been  recommended  for  discharge  and,  in 
fact,  were  not  discharged.  The  positive  predic¬ 
tive  value  of  this  test,  the  probability  that 
someone  with  a  positive  test  (Z  score  >  1.5) 
was  actually  discharged  less  than  honorably, 
is  33%. 

These  men  had  already  been  through  several 
selection  procedures  including  basic  recruit 
training,  Subscreen,  passing  Subschool,  etc. 

As  Katz  noted,  the  predictive  value  of  Sub¬ 
screen  is,  therefore,  less  in  this  group  than 
would  be  expected  if  it  had  not  been  one  of 
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the  earlier  selection  tools  employed.  The  pre¬ 
diction  value  also  would  probably  be  much 
higher  if  separate  multiple  regressions  using 
all  of  the  data,  including  demographic  infor¬ 
mation,  were  calculated  for  the  drop  catego¬ 
ries  of  interest 

To  date  the  question  of  whether  Subscreen  is 
able  to  predict  which  individuals  will  become 
psychiatric  disquals  during  active  submarine 
duty  has  not  been  investigated.  We  are  cur¬ 
rently  performing  this  research. 

Summary  and  Conclusions 

The  use  of  Subscreen  during  Basic  Recruit 
Training  for  individuals  designated  to  con¬ 
tinue  on  to  BESS  would  be  appropriate.  In 
this  context  it  would  be  used  to  screen  indi¬ 
viduals  for  referral  to  a  professional  psycholo¬ 
gist,  psychiatrist,  or  psychiatric  social  worker 
for  clinical  evaluation.  Because  it  has  subma¬ 
rine  specific  motivation  subscales  that  should 
be  used  in  the  referral  process,  it  would  only 
be  appropriate  for  individuals  scheduled  to  at¬ 
tend  BESS.  Early  identification  of  individu¬ 
als  who  are  psychiatrically  and/or 
motivationally  unsuited  for  submarine  service 
would  save  additional  training  and  travel  dol¬ 
lars. 

For  FY91  and  FY92, 124  BESS  students  were 
dropped  for  psychiatric  reasons.  Ninety-three 
of  these  were  identified  by  Subscreen.  BESS 
is  a  6  week  course  with  an  approximate  cost 
of  $3724.00  per  student  (figures  on  training 
costs  were  obtained  from  Resource/Man¬ 
power  Analysis  Course  Costing  Branch, 
NETEMPSA,  Pensacola,  Florida).  Identify¬ 
ing  these  individuals  as  unsuitable  for  subma¬ 
rine  service  resulted  in  a  cost  avoidance  of 
$346,332.00  at  BESS  alone. 

Fifteen  BESS  students  who  were  already  iden¬ 
tified  as  candidates  for  advanced  schools  were 
dropped  in  FY91  as  a  result  of  Subscreen  test¬ 


ing.  The  cost  for  these  15  students  at  Basic 
Electronic  Rate  Training  (BERT),  a  14  week 
course,  would  have  been  $12,51 1  per  student, 
a  savings  of  $187,665.00.  For  training  be¬ 
yond  BERT,  such  as  the  Radioman  pipeline, 
the  savings  per  student  would  equal  over 
$53,400.00;  a  total  of  $801, 180.  This  latter 
example  is  illustrative.  While  all  15  men 
were  scheduled  for  advanced  training,  not  all 
were  in  the  radioman  training  group.  An  addi¬ 
tional  four  of  the  psychiatric  drops  were 
scheduled  for  guided  missile  school,  and  one 
of  the  drops  was  a  corpsman  scheduled  to  go 
to  additional  training  at  the  Naval  Undersea 
Medical  Institute  (NUMI).  It  is  likely  that  at 
least  14  of  the  other  psych  drops  would  also 
have  gone  on  to  advanced  schools  at  a  conser¬ 
vative  total  cost  of  over  1  million  dollars. 

The  majority  of  BESS  graduates  go  on  to  re¬ 
ceive  additional  training  prior  to  submarine 
duty.  For  FY91  46%  of  BESS  graduates  re¬ 
ported  to  Advanced  Submarine  School  for  Ba¬ 
sic  Electronics  Rate  Training  (BERT),  13% 
reported  to  an  advanced  school  for  guided 
missile  training,  2%  reported  to  the  Naval  Un¬ 
dersea  Medical  Institute,  and  27%  received  or¬ 
ders  directly  to  a  submarine. 

Since  motivational  factors  are  not  part  of  the 
criteria  for  allowing  a  psychiatric  drop  by  the 
Medical  Department,  many  of  those  inter¬ 
viewed  are  passed  psychologically  but  are 
dropped  for  other  reasons.  Perhaps  an  "As¬ 
sessment  Center"  approach  using  information 
from  instructors,  administrators  and  medical 
(psychiatric)  personnel  would  be  useful. 

Often  information  about  an  individual  is  not 
known  by  all  concerned  in  the  evaluation 
process.  For  example,  the  clinician  may  have 
Subscreen  results  indicating  low  motivation, 
but  may  not  be  aware  of  disciplinary  prob¬ 
lems  which  could  affect  recommendation  for 
drop.  This  type  of  approach  is  feasible  for  a 
relatively  small  command  such  as  BESS,  but 
is  less  practical  for  larger  groups. 
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At  the  present,  time  Subscreen  is  not  a  useful 
measure  for  predicting  attrition  from  BESS 
for  other  than  what  are  considered  to  be  psy¬ 
chiatric  or  motivational  reasons.  Subscreen  is 
also  not  useful  for  predicting  less  than  honor¬ 
able  discharges  from  the  US  Navy.  A  combi¬ 
nation  of  Subscreen  factors  and  other 
information  such  as  AFQT,  reading  scores, 
high  school  class  standing,  etc.  might  yield  a 
better  prediction  of  non-psychiatric  attrition. 
There  is  a  long  history  of  Navy  research  using 
demographic  variables  to  predict  performance 
and  attrition.  Numerous  Navy  and  Air  Force 
studies  (Jensen,  1961;  Noddin,  1969;  Lachar, 
Sparks  and  Larsen,  1974;  Weybrew  and  Nod- 
din,  1974;  McGraw  and  Bearden,  1988;  Craw¬ 
ford  and  Fiedler,  1991)  have  identified 
motivation  as  a  prime  cause  for  unsuitable  per¬ 
formance.  Analysis  of  all  of  the  FY91  BESS 
attrition  data  showed  an  additional  16  men 
were  dropped  from  BESS  for  motivational  rea¬ 
sons.  Subscreen  may  have  identified  these 
men,  but  motivation  information  is  not  cur¬ 
rently  used  as  a  basis  for  referral  so  this  infor¬ 
mation  was  never  available  to  the  BESS  staff. 
Individuals  who  desire  to  leave  the  service  ex¬ 
press  this  interest  early  and  often.  Clearly  a 
good  test  of  motivation  would  have  great  util¬ 
ity. 

The  Navy-AFMET  test  is  currently  being 
given  by  the  Navy  as  a  general  screening  test. 
The  first  phase  of  Navy-AFMET  assesses  fam¬ 
ily,  school,  legal,  alcohol  and  anti-social  prob¬ 
lems.  It  includes  demographic  information, 
history  of  depressive  problems  and  mental 
health  treatment  history.  In  the  second  screen¬ 
ing  phase,  the  NEO  Personality  Inventory  FFI 
(Costa  and  McCrae,  1985)  is  given.  This 
phase  addresses  issues  of  neuroticism,  respon¬ 
sibility,  suicidal  ideation,  unmodulated  anger, 
perseverance,  emotional  stability,  and  self- 
confidence.  At  the  third  phase,  an  MMPI-2 
and  clinical  interview  are  given.  For  the  basic 
Navy  population  this  appears  to  be  a  thorough 
and  responsible  approach  to  general  screen¬ 


ing.  Since  it  has  only  recently  been  instituted 
(October,  1991)  at  US  Navy  Recruit  Training 
Centers  its  specificity,  sensitivity,  positive  pre¬ 
dictive  value,  etc.  in  this  population  are  un¬ 
known.  Sensitivity  of  the  Air  Force  version  of 
the  test,  AFMET,  is  low,  however  (Crawford 
and  Fiedler,  1991). 

The  submarine  environment  necessitates  addi¬ 
tional  or  revised  testing  that  increases  the  im¬ 
portance  of  claustrophobia,  sleepwalking, 
attitudes  toward  nuclear  weapons,  and  feel¬ 
ings  about  submerging  for  extended  periods. 
Claustrophobia  (situation  reaction)  alone  ac¬ 
counted  for  29  percent  of  the  Submarine 
School  psychiatric  attrition  for  FY91.  Sleep¬ 
walking  accounted  for  another  2  percent. 

These  items  are  not  adequately  covered  in  a 
general  screening  program  and  must  be  in¬ 
cluded  in  any  submarine  specific  screening 
program.  There  is  clearly  a  need  to  continue 
to  administer  Subscreen  and/or  perform  other 
submarine  specific  testing.  Additional  re¬ 
search  designed  to  improve  the  sensitivity  and 
specificity  of  Subscreen  and  to  include  moti¬ 
vational  issues  could  save  the  Navy  hundreds 
of  thousands  of  training  and  travel  dollars. 
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APPENDIX  A 


SUBSCREEN 


GDS  Classification 

EXS  Extreme  Scores 

DIS  Distortion 

GIM  Good  Impression 


Procedural  Subscales 

A  derived  score  that  includes  the  following  subscales 
(described  below)  -  GIM,  EXS,  DIS,  UCS,  PSS,  LOC, 
DEP,  SOI,  SUI,  CLA.  Different  weights  are  attached 
to  the  subscales  in  computing  this  score. 

Score  based  on  the  number  of  agree  strongly  and 
disagree  strongly  responses. 

Eight  items  related  to  traits  which  are  desirable  to 
have  but  which  are  almost  statistically  impossible  to  be 
found  in  one  individual,  e.g.,  "I  am  always  honest";  "I 
never  take  advantage  of  anyone." 

Twelve  items  which  are  socially  acceptable  and  likely 
to  be  endorsed  by  someone  trying  to  impress  people, 
e.g.,  "I  like  to  look  at  the  positive  side  of  life."  "I'm 
very  careful  in  my  work." 


Submarine  Specific  Motivation  Subscales 

MJS  Mistake  Joining  Eight  items  expressing  dissatisfaction  with  the 

Subs  submarine  force  and/or  the  Navy,  e.g.,  "It  would  take 

very  little  change  in  my  life  to  cause  me  to  leave  the 
submarine  service;"  "Joining  the  Navy  was  a  definite 
mistake  on  my  part." 

UNS  Uncertain  About  Six  items  which  reflect  ambiguous  attitudes  toward 

Subs  submarines,  e.g.,  "At  times  I'm  not  sure  I  should  have 

volunteered  for  submarines";  "I  really  don't  know  why 
I'm  interested  in  submarines." 


A-l 


COS  Conditional 

Acceptance  of 
Submarines 


UCS  Unconditional 
Acceptance  of 
Submarines 


PSM  Problems  Submerging 


PSS  Problems  Submarine 
School 


PWL  Physical  Well  Being 


LOC  Low  Situational 
Control 


ANX  Nervous  or  Worrying 


DEP  Depressed  Mood 


Five  items  expressing  acceptance  of  submarines  if 
certain  conditions  are  met,  e.g.,  "If  my  submarine  were 
to  be  stationed  outside  the  U.S.  for  long  periods  of 
time,  I'd  feel  like  quitting;"  "If  I  didn't  get  my  choice 
of  a  home  port,  I'd  feel  like  getting  out." 

Eight  items  expressing  the  acceptance  of  submarines 
even  under  negative  conditions,  e.g.,  "I  believe  that 
even  if  submariners  did  not  get  extra  pay,  I  would  still 
have  volunteered;"  "I  would  accept  almost  any  kind  of 
job  assignment  in  order  to  stay  in  the  submarine 
service." 

Four  items  related  to  working/living  under  submerged 
conditions  on  a  submarine,  e.g.,  "Working  with  the 
same  people  under  the  same  conditions  during  long 
submerged  cruises  does  not  appeal  to  me." 

Six  items  that  address  learning  skills  and  self¬ 
perception  of  capacity  to  do  well  at  submarine  school, 
e.g.,  "I  believe  Submarine  School  will  be  easy  for  me." 

Affective  Subscales 

Nine  items  related  to  the  individual's  perceptions  of 
his  health  or  stamina,  e.g.,  "My  health  is  excellent" 
(scored  negatively)  or  "I  tire  very  easily." 

A  group  of  ten  items  expressing  elements  of  worry  and 
the  feeling  that  they  have  little  control  over  the 
situation  they  are  in,  e.g.,  "I  worry  too  much  about 
things  I  cannot  change;"  "Sometimes  planning  ahead 
seems  impossible." 

Eleven  items  related  to  anxiety  and  nervousness,  e.g., 

"I  am  a  very  nervous  person;"  "Often  I  become  so 
upset  I  Find  it  hard  to  get  to  sleep." 

Eight  items  which  are  normally  associated  with 
depression,  e.g.,  "I  feel  moody  and  depressed  most  of 
the  time;  Sometimes  I  get  just  plain  sick  and  tired  of 
living." 


A-2 


Socialization  Subscales 


COE  Coercive  Attitudes  Fourteen  items  related  to  behavior  which  is  not 

normally  accepted  by  most  people  and  often  found  in 
personality  disorders,  e.g.,  "I  know  how  to  make 
people  feel  uneasy  when  I  want  to;  I  can  get  away  with 
just  about  anything  I  want  to." 

AGG  Aggressive/Destructive  Eight  items  related  to  argumentative  behavior, 

fighting,  and  anger,  e.g.,  "I  like  to  see  how  far  I  can 
push  people  before  they  fight  back;  I  have  never 
backed  down  from  a  fight." 


PHS  Problems  Home/School  Eleven  items  related  to  negative  behaviors  or 

situations  in  the  past,  e.g.,  "I  went  into  the  military 
because  I  didn't  like  my  home  life;  When  I  was  going 
to  school,  I  played  hooky  quite  often." 

SOI  Social  Isolate  Ten  items  indicating  problems  associating  with  others, 

e.g.,  "I  have  very  few  ties  to  other  people  or  places;  I 
don't  have  many  close  friends." 

IMP  Impulsive  Eleven  items  related  to  impulsive  behavior,  e.g.,  "I 

often  do  things  on  the  spur  of  the  moment  without 
stopping  to  think;  I  would  do  almost  anything  on  a 
dare." 


SSP  Social  Support  Ten  items  indicating  emotional  support  by  family  or 

friends,  e.g.,  "My  home  life  was  very  happy;  Many 
people  care  about  me." 


Additional  Subscales 


BTH 


BPC 


SUI 


Unusual  Thoughts 

Unusual  Physical 
Complaints 

Suicidal  Thoughts 


Seven  items  -  thoughts  which  most  people  do  not 
adhere  to,  e.g.,  "I  think  that  people  often  talk  about 
me  behind  my  back;  I  am  afraid  of  the  dark." 

Six  items  which  are  infrequently  endorsed  by  most 
people,  e.g.,  "I  am  almost  always  too  hot  or  too  cold;  I 
often  walk  in  my  sleep." 

Seven  items  related  to  death  or  suicidal  thoughts,  e.g., 
"I  have  sometimes  thought  of  taking  my  own  life;  I  feel 
the  world  is  not  worth  continuing  to  live  in." 


CLA  Claustrophobic  Feelings  Six  items  related  to  discomfort  in  enclosed  spaces  or 

crowds,  e.g.,  "I  often  feel  uneasy  in  a  crowd;  I  often 
feel  cramped  and  hemmed  in  when  I  am  in  a  small 
room." 


NUC  Problems  Nuclear 


DEPND  Dependency 


SCR  Self-Criticism 
COM  Competency 


Six  items  related  to  fear  or  dislike  of  nuclear  weapons 
or  radiation,  e.g.,  "I  am  concerned  about  the  possibility 
of  being  exposed  to  radiation  while  aboard  a 
submarine;  Considering  the  feelings  I  have,  I  might 
disobey  an  order  to  fire  a  nuclear  weapon." 

Eleven  items  indicating  need  for  social  support,  e.g.,  "I 
become  frightened  when  I  feel  alone;  I  am  very 
sensitive  to  others  for  signs  of  rejection." 

Nine  items  that  address  feelings  of  self-worth,  e.g.,  "I 
feel  good  about  myself  whether  I  succeed  or  fail." 

Seven  items  related  to  responsibility,  goal  setting,  and 
self-image,  e.g.,  "I  set  my  personal  goals  and  standards 
as  high  as  possible."  (Not  currently  included  in 
printout  but  discussed  in  text.) 


FACTORS 


VUL 

Vulnerability 

Thirty-three  items  which  reflect  locus  of  control, 
nervous  anxiety,  depression,  and  suicide. 

MOT 

Negative  Motivation 

Nineteen  items  that  address  attitudes  toward  the 
military  and  submarine  service. 

DPS 

Depression/Suicide 

Twenty-four  items  that  reflect  self-reported  depression 
and  suicide  ideation  and  attitude. 

SCN 

Competence 

Twenty  items  that  measure  general  competence  and 
efficacy. 

MAN 

Manipulation 

Eighteen  items  that  reflect  manipulativeness. 

ISO 

Isolation/Social  Support 

Nineteen  items  which  measure  social  isolation  and 
support. 

Other  Scores 


Problem:  Weighted  score  computed  as  follows: 

PROBLEM  =  ((-2.25  X  ZPHS)  +  (2.19  X  ZPHS2))  -  8.27 


Motivation :  Weighted  score  computed  as  follows: 

MOTIVATION  =  ((-.31  x  ZMJS)  + 

(-1.18  x  ZPSM)  + 

(.26  x  ZNUC)  + 

(-.49  x  ZUCS)  + 

(.31  x  ZPSS)  + 

(-1.60  x  ZCLA)  + 

(2.01  x  (PXM  +  CLA)) 
-2.88 


Affective:  Weighted  score  computed  as  follows: 

AFFECTIVE  =  ((-1.75  x  ZDEP)  + 

(-2.43  x  ZSUI)  + 

(3.40  x  (DEP  x  SUI)  + 
(.30  x  (ANX  x  LOC)) 
-11.45 
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Appendix  B 

Evaluation  of  Screening  Tests 


Both  a  screening  test  and  an  actual  diagnosis  can  give  either  a  positive  or  a  negative 
answer.  The  screening  test  may  or  may  not  accurately  predict  the  true  condition  of  the 
individual.  The  following  table  shows  these  possibilities: 

Diagnosis 


Condition  Present 

No  Condition 

Positive 

a 

b 

Test 

Procedure 

Negative 

c 

d 

Total 

a+c 

b+d 

The  grand  total  (n)  =  a+b  +  c  +  d 

Estimating  the  Usefulness  of  a  Test 

1.  The  sensitivity  of  a  test  is  an  estimate  of  the  probability  of  a  positive  test  result  given 
that  a  psychiatric  condition  exists. 


Sensitivity  =  a/a+c 

2.  The  specificity  of  a  test  is  the  probability  of  a  negative  test  result  given  that  no 
psychiatric  condition  exists. 


Specificity  =  d/b+d 

3.  The  positive  predictive  value  of  a  test  is  the  probability  that  a  person  with  a  positive 
test  actually  has  the  condition  in  question. 

Positive  Predictive  Value  =  a/a+b 

4.  The  negative  predictive  value  is  the  probability  that  a  person  with  a  negative  test  does 
not  have  the  condition. 


Negative  Predictive  Value  =  d/c+d 
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In  order  to  determine  a  cutting  score  for  any  test,  one  must  also  know  the  seriousness  for 
the  selection  process  of  the  two  types  of  incorrect  decisions,  false  positives,  in  this  case 
persons  recommended  for  psychiatric  evaluation  who  will  not  be  dropped,  and  false 
negatives,  people  who  should  have  been  recommended  for  evaluation  and  dropped  but 

were  not. 

5.  The  percentage  of  correct  decisions  is  the  number  of  the  cases  in  which  both  the  test 
result  and  the  diagnosis  agree. 

Percent  Correct  Decisions  =  (a  +  d/n)  x  100. 

6.  The  Coefficient  of  practical  validity  is  the  difference  between  number  of  correct  and 
incorrect  decisions. 

Coefficient  of  Practical  Validity  =  (a+d)-(b  +  c)/n 
A  high  positive  COPY  correlates  with  overall  efficiency  in  the  selection  tool. 
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